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Abstract. This paper presents a detailed proof of the triality theorem for a 
class of fourth-order polynomial optimization problems. The method is based 
on linear algebra but it solves an open problem on the double-min duality left 
in 2003. Results show that the triality theory holds strongly in a tri-duality 
form if the primal problem and its canonical dual have the same dimension; 
otherwise, both the canonical min-max duality and the double-max duality 
still hold strongly, but the double-min duality holds weakly in a symmetrical 
form. Four numerical examples are presented to illustrate that this theory can 
be used to identify not only the global minimum, but also the largest local 
minimum and local maximum. 

1. Introduction and Motivation. The concepts of triality and tri-duality were originally pro- 
posed in nonconvex mechanics [4, 5]. Mathematical theory of triality in its standard format is 
composed of three types of dualities: a canonical min-max duality and a pair of double-min and 
double-max dualities. The canonical min-max duality provides a sufficient condition for global 
minimum, while the double-min and double max dualities can be used to identify respectively the 
largest local minimum and local maximum. The tri-duality is a strong form of the triality princi- 
ple [7]. Together with a canonical dual transformation and a complementary- dual principle, they 
comprise a versatile canonical duality theory, which can be used not only for solving a large class 
of challenging problems in nonconvex/nonsmooth analysis and continuous/discrete optimization 
[7, 8], but also for modeling complex systems and understanding multi-scale phenomena within a 
unified framework [4, 5, 7] (see also the review articles [10, 12, 17]). 

For example, in the recent work by Gao and Ogden [13] on nonconvex variational/boundary 
value problems, it was discovered that both the global and local minimizers are usually nonsmooth 
functions and cannot be determined easily by traditional Newton- type numerical methods. How- 
ever, by the canonical dual transformation, the nonlinear differential equation is equivalent to an 
algebraic equation, which can be solved analytically to obtain all solutions. Both global mini- 
mizer and local extrema were identified by the triality theory, which revealed some interesting 
phenomena in phase transitions. 

The triality theory has attracted much attention recently in duality ways: Successful applica- 
tions in multi-disciplinary fields of mathematics, engineering and sciences show that this theory is 
not only useful and versatile, but also beautiful in its mathematical format and rich in connotation 
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of physics, which reveals a unified intrinsic duahty pattern in complex systems; On the other hand, 
a large number of "counterexamples" have been presented in several papers since 2010. Unfortu- 
nately, most of these counterexamples are cither fundamentally wrong (see [30, 31]), or repeatedly 
address an open problem left by Gao in 2003 on the double- min duality [9, 10]. 

The main goal of this paper is to solve this open problem left in 2003. The next section will 
present a brief review and the open problem in the triality theory. In Section 3, the triality 
theory is proved in its strong form as it was originally discovered. Section 4 shows that both 
the canonical min-max and the double-max dualities hold strongly in general, but the double-min 
duality holds weakly in a symmetrical form. Applications are illustrated in Section 5, where a 
linear perturbation method is used for solving certain critical problems. The paper ended by an 
Appendix and a section of concluding remarks. 

2. Canonical Duality Theory: A Brief Review and an Open Problem. Let us begin with 
the general global extremum problem 

(V): ext |n(a;) = W(a;) -f i(33, Aa;> - (a;,/> I a; G A'aj , (1) 

where Xa C M" is an open set, x = {xi} 6 R" is a decision vector, A = {Aij} 6 R"'^" is a 
given symmetric matrix, / = {fi} S R" is a given vector, and (*,*) denotes a bilinear form on 
R" X R" ; the function W : Xa — > R is assumed to be nonconvex and differentiable (it is allowed to 
be nonsmooth and sub-differentiable for constrained problems). The notation ext{*} stands for 
finding global extremal of the function given in {*}. 

In this paper, we are interested only in three types of global extrema: the global minimum and 
a pair of the largest local minimum and local maximum. Therefore, the nonconvex term iy(a;) in 
(1) is assumed to satisfy the objectivity condition ^, i.e., there exists a (geometrically) nonlinear 
mapping K : Xa ^ V d R™' and a canonical function V .V d R"* — > R such that 

W{x) = V(K(x)) Vtc e Xa- 

According to [7], a real valued function y : V — > M is said to be a canonical function on its effective 
domain Va C V if its Legendre conjugate V* ; V* — > R 

y*(^) = sta{($;?>-V(4)|4eVa} (2) 

is uniquely defined on its effective domain V* C V* such that the canonical duality relations 

^ = VV'(0 ^ $ = Vy*(^) ^ y(^)-f y*(?) = (^;^> (3) 

hold on Va X V*, where (*;*) represents a bilinear form which puts V and V* in duality. The 
notation sta{*} stands for solving the stationary point problem in {*}. By this one-to-one canonical 
duality, the nonconvex function W(x) = V(A(x)) can be replaced by (A(a;); ?) — y*(^) such that 
the nonconvex function Tl{x) in (1) can be written as 

H(cc,«) = (Aix);^) ~ V[<;) + ^(x,Ax) - {x,f), (4) 

which is the so-called total complementary (energy) function introduced by Gao and Strang in 
1989. By using this total complementary function, the canonical dual function 11'' : V* — > R can 
be formulated as 

n''(?) = sta{H(cc,<;)| Vcc G AfJ = U^(«) - V* (?), (5) 
where : V* — > R is called the A-conjugatc of U{x) = (x, f) — ^(x, Ax), defined by [7] as 

U''{<;) = sta{(A(£c); «) - U(a;)| x e X^. (6) 

Let cSa C V* be the feasible domain of f7^(?); then the canonical dual problem is to solve the 
stationary point problem 

(7"*): ext{nd(0|se5a}. (7) 



^The concept of objectivity in science means that qualitative and quantitative descriptions 
of physical phenomena remain unchanged when the phenomena are observed under a variety of 
conditions. That is, the objective function should be independent with the choice of the coordinate 
systems. In continuum mechanics, the objectivity is also regarded as the principle of frame- 
indifference. See Chapter 6 in [7] for mathematical definitions of the objectivity and geometric 
nonlinearity in differential geometry and finite deformation field theory. Detailed discussion of 
objectivity in global optimization will be given in another paper [18]. 



TRIALITY THEORY FOR QUARTIC POLYNOMIAL OPTIMIZATION 



3 



Theorem 2.1 (Complementary-duality principle [7]). Problem [V^) is a canonical dual to (V) 
in the sense that if (x, ?) is a critical point of E,{x,(;), then x is a critical point of {V), ^ is a 
critical point of {V^), and 

n(s) = s(s,5) = n''(?). (8) 

Theorem 2.1 implies a perfect duality relation (i.e. no duality gap) between the primal problem 
and its canonical duaP. The formulation of n''(i;) depends on the geometrical operator K{x). In 
many applications, the geometrical operator A is usually a quadratic mapping over a given field 
[7], In finite dimensional space, this quadratic operator can be written as a vector- valued function 
(see [8], page 150) 

r 1 1 

K{x) = {-x'^^'^x) : A'a C M" Va C R™, (9) 



defined by 



{B^A S E"^" is a symmetrical matrix for each k = 1,2,--- ,m, and Va C 



Va = {id IR'"| = ^x'^B'^x Va; 6 A'a, k = 1, 



In this case, the total complementary function has the form 

S (x, <;) = i {x, G (?) x) - V (?) - {x, /), (10) 
where G : M"^ j^nxn jg matrix- valued function defined by 

m 

G(?) = A + ^?fcB^ (11) 

fe=i 

The critical condition Va;H {x, ?) = leads to the canonical equilibrium equation 

Gi<;)x = f. (12) 

Clearly, for any given <; 6 V*, if the vector / £ Coi (G(<;)), where Coi (G) stands for a space 
spanned by the columns of G, the canonical equilibrium equation (12) can be solved analytically 
as** X = [G(?)]~^/. Therefore, the canonical dual feasible space cSa C V* can be defined as 

5a = {? ev: I / 6C„,(G (?))}, 

and on Sa the canonical dual n''(?) is well-defined as 

n''(0 = -i(G(«)-V,/)-v*(0. (13) 

Theorem 2.2 (Analytic solution [7]). If i & Sa is a critical solution of {T"^), then 

S = G(«)-V (14) 

is a critical solution of CP) and Tl{x) = n''(^). 

Conversely, if x is a critical solution of iV), it must be in the form of (14) for a certain critical 
solution 5 of {T"^). 

The canonical dual function n'*(<;) for a general quadratic operator A was first formulated in 
nonconvex analysis, where Theorem 2.2 is called the pure complementary energy principle, [6]. 
In finite deformation theory, this theorem solved an open problem left by Hellinger (1914) and 
Reissner (1954) (see [25]). The analytical solution theorem has been successfully applied for solving 
a class of nonconvex problems in mathematical physics, including Einstein's special relativity 
theory [7], nonconvex mechanics and phase transitions in solids [1.3]. In global optimization, 
the primal solutions to nonconvex minimization and integer programming problems are usually 
located on the boundary of the feasible space. By Theorem 2.2, these solutions can be analytically 
determined by critical points of the canonical dual function n''(i;) (see [3, 11, 14, 16]). 



■^The complementary-dual in physics means perfect dual in optimization, i.e., the canonical 
dual in Gao's work, which means no duality gap. Otherwise, any duality gap will violet the 
energy conservation law. Therefore, each complementary-dual variational statement in continuum 
mechanics is usually refereed as a principle. 

■^In this paper G~^ should be understood as the generalized inverse if det G = [8]. 
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In order to identify both global and local extrema of the primal and dual problems, we assume, 
without losing much generality, that the canonical function V : £a ^ convex and let 

S+ = e 5„ I G (^) b 0} , (15) 

S- = {<;e5JG(O-<0}, (16) 

where G (?) >z means that G (<;) is a positive semi-definite matrix and G (?) -< means that 
G (?) is negative definite. 

Theorem 2.3 (Triality Theorem [8]). Let (S,?) be a critical point o/S(a;,?). 

//G(?) 0, then ^ is a global maximizer of Problem (V^), the vector x is a global minimizer 
of Problem ("P), and the following canonical min-max duality statement holds: 

min n (a;) = H {x, ?) = max H'' (?) . (17) 

// G(?) -< 0, then there exists a neighborhood Xo x. So d Xa X Sa of {x, ?) for which we have 
either the double-min duality statement 

min n (cc) = S (S, ?) = min n'' (?) , (18) 

or the double-max duality statement 

max n (cc) = H {x, ?) = max (?) . (19) 

The triality theory provides actual global extrcmum criteria for three types of solutions to the 
nonconvcx problem {V). a global minimizer S(?) if ? S ■S^'" and a pair of the largest-valued local 
extrema. In other words, S(?) is the largest- valued local maximizer if ? S is a local maximizer; 
a;(?) is the largest-valued local minimizer if ? 6 Sa is a local minimizer. This pair of largest local 
extrema plays a critical role in nonconvcx analysis of post-bifurcation and phase transitions. 

Remark 1 (Relation between Lagrangian Duality and Canonical Duality). 

The main difference between the Lagrangian-type dualities (including the equivalent Fenchel- 
Moreau-Rockfellar dualities) and the canonical duality is the operator A : Xa — >■ Va. In fact, if A 
is linear, the primal problem (V) is called geometrically linear in [7] and the total complementary 
function S(£c,?) is simply the well-known Lagrangian and is denoted as 

L{x,<;) = {kx;^)~V*{<;)^ Fix). (20) 

In convex (static) systems, F(x) = {x, f) is linear and L{x, ?) is a saddle function. Therefore, 
the well-known saddle min-max duality links a convex minimization problem ("P) to a concave 
maximization dual problem with linear constraint: 

max{n*(?) = -y*(?)| A*? = /, ?ev:}, (21) 

where A* is the conjugate operator of A defined via (Ate;?) = (a;,A*?). Using the Lagrange 
multiplier x £ Xa to relax the equality constraint, the Lagrangian L(a;,?) is obtained. By the 
fact that the (canonical) duality in convex static systems is unique, the saddle min-max duality 
is refereed as the mono-duality in complex systems (see Chapter 1 in [7]). 

Since the linear operator A can not change the convexity ofW{x) = V{Ax), the Lagrangian 
duality theory can be used mainly for convex problems. It is known that ifW{x) is nonconvcx, then 
the Lagrangian duality as well as the related Fenchel-Moreau-Rockafellar duality will produce the 
so-called duality gap. Comparing the canonical dual function n'*(?) in (13) with the Lagrangian 
dual function n*(?) in (21), we know that the duality gap is ^{G{q)~^ f , f) . 

The canonical duality theory is based on the (geometrically) nonlinear mapping A : Xa — > Va 
and the canonical transformation W{x) = V{A{x)). The total complementary function S(£c,?) 
is also known as the nonlinear or extended Lagrangian and is denoted by L(x, ?) due to the 
geometric nonlinearity of A{x) (see [7, 10]^. Relations between the canonical duality and the 
classical Lagrangian duality are discussed in [16]. 

Remark 2 (Geometrical Nonlinearity and Complementary Gap Function). The canonical min- 
max duality statement (17) was first proposed by Gao and Strang in nonconvex/nonsmooth analy- 
sis and mechanics in 1989 [19], where Il(x) = W{x) — F(x) is the so-called total potential energy 
with W representing the internal (or stored) energy and F the external energy. The geometrical 
nonlinearity is a standard terminology in finite deformation theory, which implies that the geo- 
metrical equation (or the configuration- strain relation) ^ = A(x) is nonlinear. By definition in 
physics, a function F{x) is called the external energy means that its (sub-) differential must be the 



TRIALITY THEORY FOR QUARTIC POLYNOMIAL OPTIMIZATION 



5 



external force (or input ) f . Therefore, in Gao and Strang 's work, the external energy should be 
a linear function(al) Fix) = {x,f) on its effective domain. In this case, the matrix G(<;) is a 
Hessian of the so-called complementary gap function (i.e. the Gao-Strang gap function [19]) 

Gap{x,<;) = (-Ac{x);<;), (22) 

where Ac{x) = —^x^S'^x is called the complementary operator of a Gdteauxdifferential At {x)x = 
x^B^x of A{x) [19]. Actually, in Gao and Strang's original work, the canonical min-max duality 
statement holds in a general (weak) condition, i.e., Gap{x,^) > 0, Va; £ Xa in field theory 
(corresponding to the strong condition G(?) >z 0). The related canonical duality theory has been 
generalized to nonconvex variational analysis of a large deformation (von Karman) plate (where 
A = [34]j, nonconvex (chaotic) dynamical systems (where A = A. — d^/dt^ [10]J, and general 
nonconvex constrained problems in global optimization. Since F{x) in these general applications 
is the quadratic function — (x , Ax) + (x , f) , the Gao-Strang gap function (22) should be replaced 
by the generalized form Ga-p(x,c;) = ^{x,G((;)x) (see the review article by Gao and Sherali [17]). 
This gap function recovers the existing duality gap in traditional duality theories and provides a 
sufficient global optimality condition for general nonconvex problems in both infinite and finite 
dimensional systems (see review articles [10, 17] j. By the fact that the geometrical mapping A 
in Gao and Strang's work is a tensor-like operator, it has been realized recently that the popular 
semi-definite programming method is actually a special application (where W{x) is a quadratic 
function) of the canonical min-max duality theory proposed in 1989 (see [14, 15]J. 

In a recent paper by Voisei and Zalinescu [31], they unfortunately misunderstood some ba- 
sic terminologies in continuum physics, such as geometric nonlinearity, internal and external 
energies, and present "counterexamples" to the Gao-Strang theory based on certain "artificially 
chosen" operators A{x) and quadratic functions F(x). Whereas in the stated contexts, the ge- 
ometrical operator A should be a canonical measure ( Cauchy- Reimann type finite deformation 
operator, see Chapter 6 in [7]) and the external energy F{x) is typically a linear functional on its 
effective domain; otherwise, its (sub) -differential will not be the external force. Interested readers 
are refereed by [18] for further discussion. 

Remark 3 (Double-Min Duality and Open Problem). The double-min and double-max duality 
statements were discovered simultaneously in a post-buckling analysis of large deformed beam 
model [4, 5] in 1996, where the finite strain measure A is a quadratic differential operator from 
a 2-D displacement field to a 2-D canonical strain field. Therefore, the triality theory was first 
proposed in its strong form, i.e. the so-called tri-duality theory (see the next section). Later on 
when Gao was writing his duality book [7], he realized that this pair of double-min and double- 
max dualities holds naturally in convex Hamilton systems. Accordingly, a bi-duality theorem was 
proposed and proved for geometrically linear systems (where A is a linear operator; see Chapter 
2 in [7]j. Following this, the triality theory was naturally generalized to geometrically nonlinear 
systems (nonlinear A; see Chapter 3 in [7]) with applications to global optimization problems [8]. 
However, it was discovered in 2003 that if n ^ m in the quadratic mapping (9), the double-min 
duality statement needs "certain additional constraints". For the sake of mathematical rigor, the 
double-min duality was not included in the triality theory and these additional constraints were 
left as an open problem (see Remark 1 in [9], also Theorem 3 and its Remark in a review article 
by Gao [10]). By the fact that the double-max duality is always true, the double-min duality was 
still included in the triality theory in the "either-or" form in many applications (see [12, 16]^. 
However, ignoring the open problem related to the "certain additional constraints" on the double- 
min duality statement has led to some misleading results. 

The goal of this paper is to solve this open problem by providing a simple proof of the triality 
theory based on linear algebra. To help understanding the intrinsic characteristics of the original 
problem and its canonical dual, we assume that the nonconvex objective function W{x) is a sum 
of fourth-order canonical polynomials 

1 /I \^ 

W(x) = -J2^h[-^'^^^''^-d''j , (23) 

where B*^ = |-^i)| 6 M"''^", k = I,-- - ,m, arc all symmetric matrices, /S^ > and d'^ £ M, 
k = 1, ■ ■ ■ ,m are given constants. This polynomial is actually a discretized form of the so-called 
double-well potential, first proposed by van der Waals in thermodynamics in 1895 (see [26]), which 
is the mathematical model for natural phenomena of bifurcation and phase transitions in biology. 
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chemistry, cosmology, continuum mechanics, material science, and quantum field theory, etc. (see 
[5, 20, 23, 24]). By using the quadratic geometrical operator A{x) given by (9), the canonical 
function 

V(i) = ^i^~dff3{i~d) (24) 

and its Legendre conjugate 

y* (?) = i<;'^/3-\ + «^d (25) 

are quadratic functions, where /3 = Diag (Z?*^) represents the diagonal matrix defined by the non- 
zero vector {f^^}. 

In the following discussions, we assume that all the critical points of problem (P) are non- 
singular, i.e., if Vn(a;) = 0, then 

detV^n(S)^0. (26) 
We will first prove that if n = m, the triality theorem holds in its strong form; otherwise, the 
theorem holds in its weak form. Three numerical examples are used to illustrate the effectiveness 
and efficiency of the canonical duality theory. 

3. Strong Triality Theory for Quartic Polynomial Optimization: Tri-Duality Theo- 
rem. We first consider the case m = n. For simplicity, we assume that /3j; = 1 in the following 
discussion (otherwise, can be replaced by y/^B''' and is replaced by j ). In this 
case, the problem (1) is denoted as problem ("P). Its canonical dual is 

{V^): cxt |n'*(«) = -l/^[G(?)]-V-^«^?-«'^rf|«6 5. CM"}. (27) 

Theorem 3.1 (Tri-Duality Theorem). 

Suppose that m = n, that the assumption (26) is satisfied, that ^ is a critical point of Problem 
(V'') and that x = [G (?)]"^ /. 

If ^ & Sa , then q is a global maximizer of Problem [V^) in Sa if and only if x is a global 
minimizer of Problem (V), i.e., the following canonical min-max statement holds: 

n(S) = min n (a;) <^ max (?) = n'*(?). (28) 

On the other hand, if ^ d Sa , then, there exists a neighborhood Xo X So d K" X Sa of (S,?), 
such that either one of the following two statements holds. 
(A) The double-min duality statement 

n{x) = min n (33) <;=> min W' (?) = (?) , (29) 

or (B) the double-max duality statement 

n{x) = max n {x) <;=> max (?) = (?) . (30) 

Proof. If ? is a critical point of the canonical dual problem {T"^), the criticality condition 

[G (?)]-! BMG(?)rV 



vn'* (?) = i 



: e K" (31) 

[G (?)]-! B" [G(?)r^/ 

leads to ? = A{x). By the fact that Vn(S) = G(?)S-/ = £ K", it follows that x = [G (?)r^ / 
is a critical point of Problem CP). 

To prove the validity of the canonical min-max statement (28), let ? be a critical point and 
? S Sa . Since n'*(?) is concave on cS^, the critical point ? £ Sa must be a global maximizer of 
n'«(?) on5+. 

On the other hand, by the convexity of V (^), we have 

V(0- V(D > («-f;Vy(D) = (€-€;?>. (32) 
Substituting ^ = A [x) and ^ = A (x) into (32), we obtain 

V (A (x)) -V{A (x)) > (A {x) - A (S) ; ?>. 

This leads to 

n(x) -n(x)> (A{x) - A(S) ;?> -I- ^{x,Ax) - AS) ~ {x ~ x,f), Vx e M". (33) 
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By the fact that 

? = A(S)-d, (34) 

we have 

n(33) -n{x)> i(a;,G (?)a;) - ^(x,G{^)x) ~ (x ~ x,G{^)x). (35) 

For a fixed 5 6 cS^, the convexity of the complementary gap function Gap (a;, ?) = ^{x, G (?) x) 
on Xa leads to 

Gap{x,^)~Gaj,{x,q)>{x-X,V^Gap{x,^)) = {x~X,G{i)x) \fx £ R" . (36) 

Therefore, wc have 

n{x)-n {x) > {x~x,G (?) x) ~ {x~x,G (?) x) =0 Vtc e R". (37) 

This shows that S is a global minimizer of Problem (V). Since it is assumed that ^ 6 Sa , it 
follows that (28) is satisfied. 

We move on to prove the double-min duality statement (29). 

Let 5 be a critical point of IT'(i;) and ? £ Sa ■ It is easy to verify that 

Vn {x) = ^ Qtc^B'^a; - d''^ B'^x + Ax~f, 

V2n(s) = G(?) + F(S)F(S)^ , (38) 

where 

F{x) = [B^x,'B'^x,- - ,'B"x] . 

In light of (31), V^n'* (?) can be expressed in terms of S = [G (s)]"'^ / as follows: 

V^n'' (?) = -F (xf [G («)]-' F (5-) - I, 

where I is the identity matrix. If the critical point ? £ cS^ is a local minimizer, we have V^II'' (?) >; 
0. This leads to 

- F (S)'r[G (?)]-' F(S)b I. (39) 
Therefore, — F (S)'^ [G (?)]~^ F (S) is positive definite and F (x) is invertible. By multiplying 
^F (5;)^^ and F (x)^^ to the left and right sides of (39), respectively, we obtain 

-[G(?)]-i b (Fixfy\F{x))-\ 

According to Lemma 6.2 in Appendix, the following matrix inequality is obtained: 

V^n (S) = G (?) + F (x) F (S)^ t 0. 

By the assumption (26), x = [G (?)] ^^ / is also a local minimizer of Problem (V) ■ Therefore, on 
a neighborhood Xo x So C M" X Sa of {x, s) , we have 

min n (x) = S (x, ?) = min H'' (<;) . 

Similarly, we can show that if a; is a local minimizer of Problem (P) , the corresponding ? is also 
a local minimizer of Problem {V^). 

The next task is to prove the double-max duality statement (30). 

Let 5 S iS^ be a local maximizer of Problem {V''). Then, we have V^H'' (5) ^ 0. This gives us 

F(S)'^[G(5)r^F(S) + IbO. (40) 

Now we have two possible cases regarding the invertibility of F (x) . If F (S) is invertible, then by 
using a similar argument as presented above, we can show that the relations 

max n (a;) = H (x, = max 11'' (<;) 

hold on a neighborhood Xo X So G K" X Sa of (S,?). 

If F (a;) is not invertible, by Lemma 6.1 in the Appendix, there exists two orthogonal matrices 
E and K such that 

F(S) = EDK, (41) 

where E'^E = I = K^K and D = Diag (cri, ■ ■ ■ , (Tr, 0, ■ ■ • ,0) with ai > (T2 >■■■> err > and 
r = rank (F {x)). Substituting (41) into (40), we obtain 

-K^D^E^[G(5)]"^EDK-I^ 0. (42) 
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Thus, 

-D^ [E^G(«)Ej"^D-I^O. (43) 
Applying Lemma 6.4 in Appendix to (43), it follows that 

E^G (?) E + DD^ = E'^G (?) E + DKK^D"^ ^ 0. 

Finally, we have 

v^n (x) = G (?) + edkk'^d^e^ = G (?) + F (x) F (S)^ ^ 0. 

This means that x is also a local maximizcr of Problem (V) under the assumption (26), i.e., there 
exists a neighborhood Xo x So G X Sa of (x, ?) such that 

max n (x) = 3 (x, ?) = max H'^ (?) . 

Finally, we can show, in a similar way, that if S = [G(?)]~^ / is a local maximizcr of Problem 
(V) and ? £ Sa , the corresponding ? is also a local maximizer of Problem (V^). Therefore, the 
tri-duality theorem is proved. ■ 

Remark 4. The strong triality Theorem 3.1 can also be used to identify saddle points of the primal 
problem, i.e. ? 6 Sa is a saddle point of n''(?) if and only if a; = G(?)~^/ is a saddle point 
of n(a;) on Xa- Since the saddle points do not produce computational difficulties in numerical 
optimization, and do not exist physically in static systems, these points are excluded from the 
triality theory. 

Remark 5. By the proof of Theorem 3.1, we know that if there exists a critical point ? S Sa such 
that ? is a local minimizer of Problem (7-"*), then F(a;) must be invertible. On the other hand, if 
the symmetric matrices {S''} arc linearly dependent, then F(cc) is not invertible for any x £ M". 
In this case, the corresponding canonical dual problem (V^) has no local minimizers in Sa , and 
for any critical point ? £ Sa , the analytical solution x = [G (?)] ^^ / is not a local minimizer of 

n{x). 

4. Refined Triality Theory for General Quartic Polynomial Optimization. Let us recall 
the primal problem and its canonical dual problem in the general quartic polynomial case (n ^ m) : 

(P) : ext |n(a:) = Qa;^B''a:-d'=j + ^a;^ Aa; - a;^/ | a; e K" | , (44) 

(r"): cxt {n'*(?) = -l/^[G(?)]-V-l?^?-?^d|?e5„CM'"|. (45) 

Suppose that x and ? are the critical points of Problem (V) and Problem (T"^), respectively, where 
X = [G (?)] ^^ /. It is easy to verify that 

V2n(S) = G(?) + F(S)F(S)^ 6 M"''" (46) 

V^n"* (?) = -F (xf [G (?)]"^ F (S) - I e R'"^'". (47) 

In this case, 

F (x) = [B^x, B'^x, ■■■ , B^a;] 6 K"*"". 
To continue, we show the following lemmas. 

Lemma 4.1. Suppose that m < n. Let the critical point ? S Sa be a local minimizer of IT'(?), 
and let x = [G (?)]^"'" /. Then, there exists a matrix P S M"^"^ with rank(P) = m such that 

P^V2n(S)PbO. (48) 

Proof. By the fact that the critical point ? £ Sa is a local minimizer of n''(?), we have Vn''(?) = 
and V^W'ii) t 0. It follows that 

- F (x)'^ [G (?)]-i F (S) b I G R"^™. 

Thus, rank(F(a;)) = m. Since ? £ Sa and F (x) F (x)'^ >; 0, there exists a non-singular matrix 
T e R"^" such that 

T^G(?)T = Diag (-Ai,--- ,-A„) (49) 

and 

T^F(S)F(S)^T = Diag (ai , ■ ■ ■ , a^, 0, . . . , 0) , (50) 
where > 0, i = 1, ■ ■ ■ , n, and aj > 0, j = 1, ■ ■ ■ ,m. 
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According to the singular value decomposition theory [22], there exist orthogonal matrices U 
and E such that 



T^F(S) = U 





V 



Therefore, U is an identity matrix. Let 



/ 





V 



/dm 



; 



Then, 



v^n'^C?) = -F (S)^ [G F{S)- I 

= - (^F^t) [t^G (?) t] ^ T^F (x) - I 

= -E^RU^piag (-Ai,-- ■ ,-A„)]"^URE- I e M™-""". 
Since V'^H'' (?) 0, U is an identity matrix, and E is an orthogonal matrix, we have 

-R[Diag (Ai, • ■ ■ , A„)] R- I^xm = Diag 1, ■ ■ ■ , — 

\ Al An 

Thus, ai > Xi, i = 1, ■ ■ ■ ,m. Note that 

T'^V2n(S)T = Diag(ai-Ai,--- , a„ - A,„, -A™+i , ■ ■ ■ ,-A„). 
Let J = [Imxm, Omx(n-m)]^- Then, we have 

Am) t 0. 



1^0. 



(51) 



(52) 



J^T^V^n(S)TJ = Diag (ai - Ai, 

Let P = TJ. Clearly, rank(P) = m and P'^V^H (x) P = Diag (ai - Ai, ■ ■ ■ , - Am) t 0. The 
proof is completed. ■ 

In a similar way, we can prove the following lemma. 

Lemma 4.2. Suppose that m > n. Let x = [G{^)]^^ f be a critical point, which is a local 
minimizer of Problem CP) , where ^ Sa ■ Then, there exists a matrix Ql a W"^" mt/irank(Q) = 
n such that 

QTy2nd (^-) Q ^ 0. (53) 

Let pi, ■ ■ ■ , pm be the m column vectors of P and let qi, • • • , be the n column vectors of 
Q, respectively. Clearly, pi, ■ ■ ■ , Pm are ra independent vectors and qi, ■ ■ • , q^ are n independent 
vectors. We introduce the following two subspaces 

A't = {a; 6 I a: = S + eipi + •■■ + 6»mpm, 6»i e K, i = 1, ■■■ ,m}, (54) 
5b = {<; e I <: = 5 + i?iqi +••• + *uq„, i3i e M, i = !,■■■ ,n}. (55) 

Theorem 4.3 (Refined Triality Theorem). 

Suppose that the assumption (26) is satisfied. Let ^ be a critical point of n''(i;) and let x = 

[G m-'f. 

If <; & Sa , then the canonical min-max duality holds in the strong form: 



Tl(x) = min El (a;) 



max n"* (?) = n'*(«). 



(56) 



If i (H Sa , then there exists a neighborhood Xo X So C K" X Sa of {x,i;) such that the 
double-max duality holds in the strong form 



n(a;) = max H (x) ■ 



max n-^" (?) = (?) . 



(57) 



However, the double-min duality statement holds conditionally in the following super- symmetrical 
forms. 
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1. If m < n and <; G Sa is a local minimizer o/ !!'*(<;), then x = [G (^)] ^ f is a saddle point of 
Tl{x) and the double-min duality holds weakly on Xo H X\, X So, i-e. 

n{x) = mill n (x) = mill U"^ (q) = (58) 

2. If m > n and x = [G(^)]~"'" f is a local minimizer ofIl(x), then ^ is a saddle point o/n''(^) and 
the double-min duality holds weakly on Xo X So r\ S\, , i.e. 

n{x) = min n(x) = min (<;) = n''(«). (59) 

Proof. The proof of the statements (56) and (57) are similar to that given for the proof of 
Theorem 3.1. Thus, it suffices to prove the double-min duality statements (58) and (59). 
Firstly, we suppose that m < n and ^ is a local minimizer of Problem {V^). Define 

fihr ■ ■ ,tm) = n(S + tlSi H h tmXm). (60) 

From (47), we obtain 

-F (x)"^ [G («)]-^ F (S) b I G R"^'". 

Thus, F (xy^ [G (t)]""' F (x) is a non-singular matrix and rank (F(S)) = m < n. We claim that 
X = [G (?)]~ / is not a local minimizer of Problem (P) . On a contrary, suppose that x is also a 
local minimizer. Then, we have 

v^n {x) = G (?) -I- F (x) F (S)^ y 0. 

Thus, F {x) F {xf > -G (?) . Since ? 6 iSa and rank (F) = m, it is clear that 

n = rank (G (?)) = rank {f (x) F (S)^) = m. 

This is a contradiction. Therefore, x = [G (5)] ^^ / is a saddle point of Problem (V). 

It is easy to verify that n(S) = n'*(^). Thus, to prove (58), it suffices to prove that g R™ is 
a local minimizer of the function (p{ti, - ■ ■ ,t„). 

It is easy to verify that 

v</5(o,--- ,0) = [i\/n{x)fpi,--- ,i\7n{x)fpmf = vn(s))^p = o (ei) 

and 

V^(^(0,--- ,0) =P'^v2n(S)P. (62) 

In light of Lemma 4.1 and the assumption (26), it follows that £ is, indeed, a local minimizer 
of the function if>{ti, ■ ■ ■ , tm). 

In a similar way, we can establish the case of m > n. The proof is completed. ■ 

Remark 6. Theorem 4.3 shows that both the canonical min-max and double-max duality state- 
ments hold strongly for general cases; the double-min duality holds strongly for n = m but weakly 
for n ^ m in a symmetrical form. The "certain additional conditions" are simply the intersection 
Xo n Xi, for m < n and So f] S\, for ni > n. Therefore, the open problem left in 2003 [9, 10] is 
solved for the double- well potential function W{x). 

The triality theory has been challenged recently in a series of more than seven papers, see, 
for example, [31, 32]. In the first version of [32], Voisei and Zalinescu wrote: "we consider that 
it is important to point out that the main results of this (triality) theory are false. This is done 
by providing elementary counter-examples that lead to think that a correction of this theory is 
impossible without falling into trivia" . It turns out that most of these counter-examples simply use 
the double-well function W{x) with n ^ m. In fact, these counter-examples address the same type 
of open problem for the double-min duality left unaddressed in [9, 10]. Indeed, by Theorem 4.3, 
we know that both the canonical min-max duality and the double-max duality hold strongly for 
the general case n ^ m. However, based on the weak double-min duality, one can easily construct 
other V-Z type counterexamples, where the strong double-min duality holds conditionally when 
n ^ m. Also, interested readers should find that the references [9, 10] never been cited in any one 
of their papers. 
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5. Numerical Experiments. In this section, some simple numerical examples are presented to 
illustrate the canonical duality theory. 

Example 1 (m = n = 2). Let us first consider Problem (V) with n = m = 2. 



ext < n (a;) 



where 



-x'^B^x - d2 



ai 







■ fei ■ 


, b2 = 











a2 












b2 



-x"^ Ax ~ x'^ f\ X 6 ' 



/=[/l,/2]' 



(63) 



The canonical dual problem can be expressed as 



Thus, 



and 



ai + ftfei 



vn-* (<;) 



a2 + f2te 



■ (t? +?!) - (rflfl +'i2T2). 



i>2/- 



■f2-l 



-fe2/2{ai+?ibi)' 



1 



1 



-bi/|(a2 +?262)" 

Now, we take bi = b2 = /i = /2 = 1> (ii = ^2 = 1 and ai = —2, 02 = —3. It is easy to 
check that II'* (?) has only one critical point ti = (^1 + 3.35991198^ in cS^ and four critical 

points ?2 = (3/2, 2.59827880) , ^3 = (l - v^, -0.45819078^ , ?4 = (l - v^, 2.5982788o) , ^5 = 

(3/2,-0.45819078) in Sa , respectively. Furthermore, ^2 is a local minimizer and 53 is a local 
maximizer; the solutions ?4 and ^5 are saddle points of Tl'^ (?) in Sa ■ Thus, by Theorem 3.1, we 
know that xi = (2.41421356, 2.77845711) is a global minimizer, while X2 = (-2, -2.48928859) is 
a local minimizer and 0:3 = (—0.41421356, —0.28916855) is a local maximizer. The corresponding 
values of the cost function are 

n (Si) = -14.0421 < n {x2) = -4.3050 < n (S3) = 0.5971. 

The graph of EI (x) and its contour are depicted in Figure 1. 





4 -4 -2 2 4 

Figure 1. Graph of n(a;) (left) and contours of n(a;) (right) for 
Example 1 



Example 2 (n = 2,m = 1). We now consider Problem (P) with n = 2, m = 1, A = 
Diag (-0.2, -0.8), B = Diag (1, 1), / = (0.9, 0.3)^, d = 4. Then, its dual problem is 

2 V-0.2-(^? -0.8 + ?/ 2 

We can verify that H"^ (?) has one critical point ?~i = —0.90489505 in Sa and two critical points <^ = 
—0.12552589 and ?3 = —3.974788888 in Sa . Furthermore, ?2 is a local minimizer and q is a local 
maximizer of n''(?). According to Theorem 4.3, Si = (A + ?-iB)-l/ = (1.27678581, 2.86000142)^ 
is the unique global minimizer of U(x), S2 = (A + C2B)-^f = (-2.76475703, -0.32414004)^ is 
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a saddle point and X3 = {A + <;~3B)-^f = (-0.21557976, -0.06283003)^ is a local maximizer 
of Il(x). Let p = (1,0)^ and <p{0) = Il{x2 + Op). Then, it is easy to verify that there exists 
neighborhoods Xo C K and iSo C M such that & Xo, C2 & So and 

min = min El'^f?). 

This example shows that even if n > m, the canonical min-max duality and the double-max 
duality still hold strongly. However, the double-min duality statement should be refined into an 
m— dimensional subspace in this case. 

Example 3 (n = l,m = 2). We now consider Problem [V) with n = 1, m = 2, A = —0.2, 
= 0.3, = 0.7, di =3, d2 = 2.7 and / = 1.4. Then, its dual problem is 



1 



iiB^ 



1 



We can verify that (?) has one critical point = (-0.35012607, 3.48303916)^ in SJ and two 
critical points ^2 = (-2.98705125,-2.66978626)"^ and ?3 = (-0.70765026,2.64881606)^ in Sa ■ 
Furthermore, t2 is a local maximizer and ?3 is a saddle point of !!''(<;). According to Theorem 
4.3, XI = G(«i)~V = 4.20307342 is the unique global minimizer of n(3;), X2 = G(^2)~V = 
-0.29381114 is a local maximizer of n{x). Let q = (1, 0)^ and i/)(i?) = n''{i3 + i?q). Then, it is 
easy to verify that there exists neighborhoods A'o C M and iSq C K such that 0:3 £ A'o , S iSo and 

min Tl{x) = min 

This example shows that if n < m, the canonical min-max duality and the double-max duality 
still hold strongly. However, the double-min duality statement should be refined into an n— 
dimensional subspace in this case. 

Example 4 Linear Perturbation. Let us consider the following optimization problem without 
input (/ = 0) 



(V2) ■■ exti U{x) 



■ (xi -I- X2) 



(xi - X2) 



I a; 6 



Problem {'P2) has four global minimizcrs xi = (1, 0) , 0:2 = 
the optimal cost value is 0. Its canonical dual problem is 



(0, -1) , X3 = (0, 1),X4 = (-1, 0) and 



H'* (?) : 



n'' (^) has only one critical point <; = {~^>~^) 6 Sa ■ Furthermore, we can check that x = 
[G (^)] ~^ / = (0, 0) is a local maximizer of Problem (7^2) ■ Thus, we cannot use the canonical dual 
transformation method to obtain the global minimizer of Problem {'P2) since this problem is in a 
perfect symmetrical form without input, which allows more than one global minimizer. Now we 
perturb Problem {'P2) as follows. 



ext.. 



2H2 (a;) 



1 



■ (xi 4- X2) 



1 



(xifl -I- X2f2) . 



Its canonical dual function is expressed as 



(?) : 



1 



8<;if2 



[dl + ?2) (/? + /I) + 2 (?1 - f2) /1/2] - - (?? + 4) - (?1 + ?2) . 



0.005 and solving VHf (5) 



0, the results obtained are listed in Table 



(0.00299107, 0.00199602) e St and (?) = -0.00500648. Thus, 



Taking /i = 0.001, /2 = 
1. We can see that 5 

X = [G(s)]"^ / = (0.000495793, 1.00249) is the global minimizer of Problem Clearly, this 

X is very close to X3. If we take /i = 0.001, /2 = —0.005, the global minimizer of Problem 
is S = [G (^)]~^ / = (0.000496288, -1.00249) which is close to X2 = (0, -1). This example shows 
that if the canonical dual problem has no critical point in Sa , a linear perturbation could be used 
to solve the primal problem. 
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f = (';i,?2) 


X = {xi,X2) 


G(<;) 


The primal problem 


(-0.49982, -0.499992) 


(-0.00100008, -0.00500014) 


G -< 


local max 


(-0.00300907, -0.499992) 


(-0.496493, -0.500493) 


G -< 


saddle point 


(0.00299107, -0.499992) 


(-0.499493, -0.503493) 


G -< 


saddle point 


(-0.49982, -0.00200402) 


(0.495997, -0.501997) 


G -< 


saddle point 


(-0.00300907, -0.00200402) 


(0.000504216, -0.99749) 


G -< 


local min 


(0.00299107, -0.00200402) 


(1.00049,0.00249576) 


indefinite 


saddle point 


(-0.49982,0.00199602) 


(-0.503997,0.497997) 


indefinite 


local max 


(-0.00300907,0.00199602) 


(-0.99949,0.00250409) 


indefinite 


saddle point 


(0.00299107,0.00199602) 


(0.000495793, 1.00249) 


G >- 


global min 



6. Appendix. In this Appendix, we present several lemmas which are needed for the proofs of 
Theorem 3.1 and Theorem 4.3. 

Lemma 6.1 (Singular value decomposition [22]). For any given G £ R"^" with rank(G) = r, 
there exist U e IR">'", D e IR"^'" and R e such that 

G = UDR, 

where U, R. are orthogonal matrices, i.e., U^U = I = R-^R, and D = Diag (ci, • • ■ , cFr, 0, ■ ■ • ,0), 

CTl > (T2 > ■ ■ ■ > 0"i- > . 

Lemma 6.2. Suppose that G and U are positive definite. Then, G >; U i/ and only j/U~^ >; 
G-i. 

Proof. The proof is trivial and is omitted here. 

Lemma 6.3 (Proposition 2.1 in [21]). For any given symmetric matrix M expressed in the form 

Mil Mi2 
M21 M22 

such that M22 >- 0. Then, M ^ «/ and only ifMn - Mi2M22^M2i b 0. 

The following lemma plays a key role in the proof of Theorem 3.1 and Theorem 4.3. 
Lemma 6.4. Suppose that P e R"X", U e R^x™, and D e R"^™, Furthermore, 



M ■- 



D 
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r X (m™r) 



{n — r)xr ^ (rt — r) X (m — r) 



where Dn £ R''^'" is nonsingular, r = rank(D), and 

Uii 



Pii P12 
P21 P22 



^ 0, U 







(m — r) X r 



X (m — r) 

U22 



^0, 



Fij and IJa, i,j = 1,2, are of appropriate dimension matrices. Then, 
P + DUD^ ^ <^ -D^P^^D - U"^ d 0. 
Proof. Suppose that P + DUD^ ^ 0. Then, 

P DUD^ 



(64) 



-Pii - DiiUiiDfj -P12 
-P21 -P22 



>- 0. 



Since P = 
inequality 



Pii P12 
P21 P22 



-< 0, it follows that — P22 0. By Lemma 6.3, we have the following 
- Pii - DiiUiiDfi + Pi2P22^P2i t (65) 
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which leads to 

-Pll + Pl2P22^P21 b DiiUiiDfj. 
Since P ^ and U ^ 0, it follows from Lemma 6.2 that 

(-Pll +Pl2P22'P2l)"' d (Dfi)"'u-lD-l. 

Thus, 

Dfi (-Pll +Pi2P22'P2i)"'dii d U-/. (66) 

Note that 

p„i ^ [ (Pii - P12P2VP21)"' Pri'Pi2 (P2iPri'Pi2 - P22)~' " 

[ (P2iPj:i'Pi2 -P22)"'P2lPn (P22 -P2lP3:i'Pl2) 

By virtue of Lemma 6.3, we obtain 

■ Dii 


i.e., the right hand side of (64) holds. 

In a similar way, we can show that if -D^P'^D-U-i ^ 0, then P + DUD'^ ^ 0. The proof 
is thus completed. ■ 



Dii 




uri 




u 



7. Conclusion Remarks. In this paper, we presented a rigorous proof of the double-min duality 
in the triality theory for a quartic polynomial optimization problem based on elementary linear 
algebra. Our results show that under some proper assumptions, the triality theory for a class 
of quartic polynomial optimization problems holds strongly in the tri-duality form if the primal 
problem and its canonical dual have the same dimension. Otherwise, both the canonical min-max 
and the double-max still hold strongly, but the double-min duality holds weakly in a symmetric 
form. 
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